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ABSTRACT 



A method and apparatus for recording audio so that the 
recording can be authenticated as to both content and time 
of recording is provided. The system may be implemented as 
a central server that is accessed via one or more telephone 
lines, or as a stand-alone unit. The system operates by 
encrypting audio information, storing the encrypted 
information, and providing users with a cryptographic key 
that can be used to decrypt the stored information. 
Preferably, time stamps are embedded in the stored infor- 
mation. Digital signatures may be used to provide additional 
security. 

3 Claims, 15 Drawing Sheets 
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METHOD AND APPARATUS FOR THE 
SECURE STORAGE OF AUDIO SIGNALS 

BACKGROUND OF THE INVENTION 

This invention relates to a method and apparatus for 
recording audio information in an authenticatable, tamper- 
proof manner. 

Traditionally, written documents have been used to pro- 
vide permanent records of transactions and agreements. One 
example of this type of document is a contract for the sale 
of an item, which typically identifies the name of the parties, 
the date, the subject matter of the contract, and a price. The 
contract provides a permanent record that can be used at a 
later date to establish the terms of the agreement between the 
parties. 

Oral contracts, on the other hand, do not provide a 
permanent record of the terms of the agreement. As a result, 
if a dispute arises over the terms of the agreement at a later 
date, it becomes difficult to prove exactly what the parties 
agreed to, or whether they made a binding contract at all. 
Because there is no permanent record, an unscrupulous party 
could be untruthful about the agreed-upon terms to escape 
his obligations. Even absent dishonesty, parties to an oral 
contract may have different recollections of exactly what 
they agreed to. Moreover, one of the persons who entered 
into the agreement may be permanently or temporarily 
unavailable. These problems tend to worsen as time passes. 

Because of these problems, all states have statutes declar- 
ing that certain oral agreements are unenforceable, typically 
including the sale of land, and the sale of goods exceeding 
a certain value. If a trustworthy record of an oral agreement 
or transaction could be obtained, however, the problems of 
oral agreements could be overcome. 

Existing methods of recording conversations, however, do 
not address these problems. For example, telephone answer- 
ing machines, tape recorders, and handheld digital audio 
recording devices can be used to record a voice or a 
conversation. It is, however, relatively easy to delete or to 
alter the recorded audio information. In particular, readily 
available electronic devices can splice sections out of an 
audio conversation, and can even rearrange words to make 
it appear that a party said something that was never actually 
said. Moreover, there is no effective way for parties to sign 
an audio recording. As a result, it may be difficult to identify 
the parties that actually agreed to the terms contained in an 
audio conversation and intended to abide by such terms. 
Further, there is no way known to applicants to verify that 
an oral negotiation matured into an agreement. 

In addition, existing telephone answering machines and 
tape recorders do not provide a reliable indication of when 
the conversation occurred. While some answering machines 
do record the time a call was received, this "time stamp" is 
extremely unreliable because a party could re record a new 
time over the time recorded by the answering machine. 
Alternatively, a party might either intentionally or acciden- 
tally set the date on an answering machine incorrectly. This 
would allow two corroborating parties to pretend that they 
made an agreement on a certain date, even though the 
agreement was not made until a later date. As a result, 
telephone answering machines and ordinary cassette record- 
ers do not alleviate the problems of oral agreements 
described above. 

STEN-TEL is an example of a system designed specifi- 
cally for recording telephonic audio information. STEN- 
TEL is available from Sten-tel Inc. (having a place of 
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business at 66 Long Wharf, Boston, Mass. 02110). To use 
STEN-TEL, a person places a telephone call to the STEN- 
TEL server, and the server digitally records the telephone 
call. After the digital recording is made, a transcriptionist 

5 accesses the recording and generates a typed record of the 
telephone call. The typed transcription is then uploaded to 
the server, where it is stored. Permanent storage of the 
digitally recorded audio conversation is optional. After the 
transcription is stored in the server, it can be downloaded to 

10 the users. Every transcription is assigned a unique identifi- 
cation number, and all status information is maintained in a 
centralized database. 

The STEN-TEL system does not, however, overcome the 
drawbacks of existing telephone answering machines and 

15 audiocassette recorders. First, the ability to restrict access to 
files is limited or non-existent. Apparently any person who 
has the file identifier can access the stored information. 
Second, the information is vulnerable to tampering. Third, 
although STEN-TEL apparently stores the time of the call, 

20 time stamps are not embedded into the stored information. 
This makes STEN-TEL vulnerable to modifications of the 
stored date for a given conversation. Finally, digital signa- 
tures are not used to provide security and/or authenticate the 
parties. 

25 One system that does incorporate certain security features 
is described in U.S. Pat. No. 5,594,798 (Cox et aL), which 
describes a voice messaging system. In Cox's system, 
however, an encryption key is stored along with the 
encrypted message. Because a hacker could obtain access to 

30 the encrypted message by retrieving the encryption key, 
Cox's system is vulnerable to attack. In addition, Cox's 
system is intended for use with secure telephone devices 
(STD). Ordinary telephones cannot call into Cox's system to 
have an audio message recorded. 

35 No existing audio recording system is known to appli- 
cants that facilitates the permanent recording of an audio 
conversation in an authenticatable form so that a user can 
simply place a telephone call to a central server and have the 
server encrypt the conversation and record the time of the 
conversation, all in a tamper-proof manner. 

SUMMARY OF THE INVENTION 

This invention advantageously provides a user-accessible 

45 system that can record audio conversations in a secure 
manner, whereby both the content and the time of the 
conversation are authenticatable. 

One aspect of the invention provides an apparatus and a 
corresponding process that includes a signal-receiving 

50 interface, an encryption processor for encrypting the 
received signals, and a storage device for storing the 
encrypted signals. A crypto-key generator generates and 
transmits a crypto-key, and a message ID generator gener- 
ates and transmits a message ID. A database stores the 

55 message ID so that it is associated with the stored signals. 
Another aspect of the invention provides an apparatus and 
a corresponding process that includes an audio signal receiv- 
ing interface for receiving audio signals from two sources, 
an encryption processor for encrypting the received audio 

60 signals, and a storage device for storing the encrypted 
signals. A crypto-key generator generates and transmits a 
crypto-key to two destinations, and a message ID generator 
generates and transmits a message ID to the two destina- 
tions. A database stores the message ID so'that it is associ- 

65 ated with the stored signals. 

Another aspect of the invention provides an apparatus and 
a corresponding process that includes an interface for rcceiv- 
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ing audio signals from two sources, an encryption processor 
for encrypting the received audio signals, and a storage 
device for storing the encrypted audio signals. A crypto-key 
generator generates two crypto-keys and transmits them to 
two destinations, respectively. A message ID generator gen- 
erates and transmits a message ID to the two destinations. A 
database stores the message ID so that it is associated with 
the stored signals. 

Another aspect of the invention provides an apparatus and 
a corresponding process that includes a number of audio 
signal receiving interfaces, and an encryption processor for 
encrypting the audio signals arriving from those interfaces. 
Encrypted audio signals, corresponding to a time during 
which a given one of the audio signal receiving interfaces is 
active, are generated and stored. Crypto-keys and message 
IDs are generated and distributed for each stored signal. A 
database stores the message ID so that it is associated with 
the stored signals. 

Another aspect of the invention provides a system and a 
corresponding process that establishes an audio connection 
with a calling party, receives an audio communication from 
the calling party, and encrypts the audio communication. 
The encrypted audio communication is stored, and a code 
for decrypting the encrypted audio communication is pro- 
vided to the calling party. 

Another aspect of the invention provides a system and a 
corresponding process that establishes an audio connection 
with at least two parties, accesses an audio communication 
between the parties, and encrypts the audio communication. 
A key, which can be used to decrypt the encrypted audio 
recording, is generated. At least two access codes are also 
generated; any of which can be used to obtain access to the 
encrypted audio recording. The key is transmitted to all the 
parties, and one of the access codes is transmitted to each 
party so that each party receives a unique access code. 

Another aspect of the invention provides a system and a 
corresponding process that includes means for establishing 
an audio connection with the parties, means for accessing an 
audio communication between the parties, and means for 
encrypting the audio communication. At least two keys are 
generated; any of which can be used to decrypt the encrypted 
audio recording. They are transmitted to the parties so that 
each party receives a unique key. 

Another aspect of the invention provides a process that 
includes the steps of establishing an audio connection 
between at least two parties and a remote recording device, 
and transmitting an audio communication between the par- 
ties. The process also includes the steps of receiving from 
the recording device, for each of the parties, a message ID, 
a cryptographic key, and one of a plurality of access codes. 
These items are required for future playback of the audio 
communication recorded by the remote recording device. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1A is a block diagram depicting the flow of infor- 
mation for recording audio information from two parties in 
accordance with this invention. 

FIG. IB is a block diagram depicting the flow of infor- 
mation for playing back previously recorded information 
from two parties in accordance with this invention. 

FIG. 1C is a block diagram depicting the flow of infor- 
mation for modifying or deleting previously recorded infor- 
mation from two parties in accordance with this invention. 

FIG. ID is a block diagram depicting the flow of infor- 
mation for recording audio information from one party in 
accordance with this invention. 
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FIG. IE is a block diagram depicting the flow of infor- 
mation for playing back previously recorded information 
from one party in accordance with this invention. 

FIG. 2 is a block diagram showing a prefe rred e rnbod i- 
5 ment of a server in accordance with this invention. 

FIG. 3 is a table describing a preferred set of fields for the 
message database of FIG. 2. 

FIG. 4 is a table describing a preferred set of fields for the 
0 message access database of FIG. 2. 

FIG. 5 A is a table describing a preferred set of fields for 
the caller database of FIG. 2. 

FIG. 5B is a table describing a preferred set of fields for 
the message archive of FIG. 2. 
15 FIG. 6 is a high-level flow chart depicting the recording 
of audio information. 

FIGS. 7A an 7B are flow charts depicting the processing 
of an incoming call and the establishing of connections 
between the central server and the parties. 

FIG. 8 is a flow chart depicting the pre-recording pro- 
cesses performed in accordance with the present invention. 

FIG. 9 is a flow chart depicting the recording of audio 
information in accordance with the present invention. 
25 FIG. 10 is a flow chart depicting the end of the recording 
process in accordance with the present invention, including 
the distribution of the cryptographic keys. 

FIG. 11 is a time line depicting participation in a multi- 
party call being recorded in accordance with the present 
30 invention. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENT 

FIGS. 1A-1E are included to describe the high-level 

35 operation of the invention, further details of which are set 
out herein below. 

FIG. 1A is a system overview of an audio recording 
system that applicants refer to as the "audio vault". This 

4Q first-described embodiment is used to record a conversation 
between two parties 13 and 14 linked to an audio vault 12 
via a najrerf conventioj iaUelepbone. connections 15 and 16. 
A firsttelephone connection 15 is established with the first 
13 party and a second telephone connection 16 is established 

45 with the second party 14. The audio vault. conferences both 
telephone connections 15 and 16 together soTHaTTfie two 
parties 13 and 14 can converse with one another. Because 
two telephone connections 15 and 16 are used, the audio 
vault can send individual information to each party without 

_ o revealing that information to the other party. 

As the parties converse with each other, the audio vault 12 
monitors the call, digitizes and encrypts the conversation, 
and records the encrypted information. A message ID, a 
crypto-key, and access codes are distributed to the parties for 

55 subsequent use for playback. 

FIG. IB depicts the standard playback mode of the audio 
vault. In this mode, any party, 13 or 14 that participated in 
the original conversation can play back the recorded con- 
versation. The party calls up the audio vault, using an 

60 ordinary telephone connectiorrtS, and provides the message 
ID number, the crypto-key, and their individual access code. 
The audio vault then decrypts the stored information using 
the crypto-key and plays the decrypted conversation back to 
the party. 

65 In an alternative embodiment, in lieu of sending the audio 
information over ordinary telephone lines, the participating 
party can access the information via computer network 
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connections or the Internet. In the latter case, the party 
woulci provide Ihe required access code and crypto-key via 
a website. The audio information would then be downloaded 
to a terminal. 

FIG. 1C depicts a second playback mode in which modi- 
fication of the recorded information can be affected. In 
accordance with a feature of the invention, authorization 
from both of the parties to the conversation must be received 
before the audio vault can modify stored information. Thus, 
the audio vault can modify or delete a call only if both 
parties 13 and 14 provide their individual access codes to the 
audio vault 12 via the respective telephone connections 15 
and 16, and at least one of the parties 13 or 14 also provide 
the message ID number and the crypto key. The audio vault 
can not modify a recording without authorization from all of 
the parties to the recording. 

While the embodiment described above works with two 
users 13 and 14 linked to the audio vault 12 using respective 
telephone connections 15 and 16, the embodiment of the 
audio vault shown in FIG. ID operates to record audio input 
from a single telephone line. In this embodiment, the audio 
vault can serve a single user at a remote location. 
Alternatively ^ this embodi ment rjifly»bffi«Ufiod.ilo w rfrorH-fb 

In this embodiment, the audio vault 12 receives the audio 
information on a single connection and provides the calling 
party(s) with a single message ID and a crypto-key that can 
be used to retrieve the audio information. 

In yet another embodiment, also represented by FIG. ID, 
the audio vault is configured as a stand-alone unit and is not 
connected to a telephone line. In this embodiment, the audio 
vault is similar to a tape recorder or a "Voice -It" handheld 
digital audio recorder, but adds cryptography and time 
stamping to provide security, as explained further below. 35 

FIG. IE represents the playback of information from the 
embodiments of the audio vault shown in FIG. ID (i.e. the 
single line telephone embodiment or the stand-alone 
embodiment). When the party 13 provides the message ID 
and the crypto-key, the audio vault 12 will decrypt and plays 40 
back the recorded audio information. 

FIG. 2 is a block diagram of one embodiment of audio 
vault 12. The audio vault includes a CPU 21 that performs 
the processing functions. It also includes a read only 
memory 22 (ROM) and a random access memory 23 
(RAM). The ROM 22 is used to store at least some of the 
program instructioas that are to be executed by the CPU 21, 
such as portions of the operating system or BIOS, and the 
RAM 23 is used for temporary storage of data. A clock 
circuit 24 provides a clock signal, which is required by the 50 
CPU 21. The interconnection and function of a CPU in 
conjunction with ROM, RAM, and a clock circuit is well 
known to those skilled in the art of CPU-based electronic 
circuit design. 

The audio vault 12 further includes a communication port 55 
25 connected to CPU 21 that enables the CPU 21 to 
communicate with devices external to the audio vault. In 
particular, the communication port 25 facilitates communi- 
cation between call center interface (CCI) 26 and the CPU 
21, so that information arriving from the CCI 26 can be 60 
processed by the CPU 21, and the CPU 21 can send 
information to users via the CCI 26. Preferably, the CCI 
includes a private branch exchange (PBX) 26a that can 
switch multiple telephone lines, an automatic call distributor 
(ACD) 266, and an interactive voice response unit (IVRU) 65 
26c connected in a manner well known to those skilled in the 
art of telephone communications. 
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CPU 21 can also store information to, and read informa- 
tion from, a data storage device 27, such as a magnetic, 
optical or equivalent type storage device. This data storage 
device 27 includes a message database 27a, a message 
access database 27/?, a caller database 27c, and a message 
archive 21d, which are described below. Optionally, any of 
the information that is stored in the data storage device 27 
may also be stored at a remote location (not shown) to 
provide a back-up version in the event of data loss. 

The program that is executed by the CPU 21 (the opera- 
tion of which is described below) may be stored in the data 
storage device 27, the RAM 23, or the ROM 22. This 
program controls the operation of the CPU 21, which in turn 
controls the operation of the audio vault. 

A cryptographic processor 28 is connected to CPU 21 and 
data storage device 27. This cryptographic processor 28 has 
the capability of generating crypto -keys, and encrypting and 
decrypting information. 

FIG. 6 is a high-level flow chart depicting the operation 
of the audio vault. The process begins in step S10 when a 
first party (party A) places a call to the audio vault. This 
incoming call is handled by PBX 26a and the associated 
ACD 26b. The call can be a "900" number call, with billing 
services provided by the service provider, as is known 
conventionally with 900 number services. Alternatively, the 
call can be to a conventional direct dial number or an 800 
number, and a customer's credit card account can be charged 
for the service. Other billing options for the use of the audio 
vault include, for example, a flat monthly fee, billing by a 
long distance service provider, prepaid phone cards, and the 
like. 

After the first party has established a connection with the 
audio vault, a connection with the second party (party B) 
must be established. This is depicted in step S12 where the 
audio vault 12 places a call to party B. In this case, the 
telephone number of party B is provided to the audio vault 
12 by calling party A. Alternatively, instead of having the 
audio vault initiate the call to party B, party B may call in 
to the audio vault. In this case, party B provides an identi- 
fication number or password, pre-established between par- 
ties A and B, that enables the audio vault to match party B's 
call to party A*s call, with the audio vault making the 
connection between the two calls. 

After the connections are established between the audio 5 
vault 12 and both parties, the audio vault conferences the |j 
two calls together, which provides an audio connection 
between the parties. This allows the parties to talk directly 
to each other. At this point nothingis being.re corded without 
the cons ent oCajllfeirtes, to Jhe. conversatio n . If, during the 
conversation, the, p arties jgregjo reco rd their conversation 
(as shown in step S14), one of th£partiesjsends a recording*"* 
req uest to the audi ovault^ (as shown in step S16). THis can*^ 
be accomplished, for example, by pressing one or more 
buttons on a touch tone telephone. Alteiratively^e-^wHo*-* 
vauAM aWKHW ^^jf^ amm 
usu^a^voke^feca^tion^ 
re qu£sJJ5^ecQrc^ 

to initiate the recording may be accomplished using any 
number of standard techniques well known to those skilled 
in the art. Next, in step SI 8, the audio vault 12 asks for 
verification of permission to record from the other party. For 
example, the audio vault 12 may generate an mdin.fitatnmftnt 
que r ^ g"l1 ie w sg CoTR t*p^ f rt y" <s D o * y ou i * d gi co Mt>»begtii * TefO 'rd- 

keys or with a voice command, and the response is inter- ~ 
prcted in the same way as the initial request to start record- )j 
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Next, in step S20, the audio vault 12 records the phone 
conversation and stores it in the data storage device 27. The 
conversation is digitized and encrypted by cryptoprocessor 
28, so that it can only be decrypted using the appropriate 
decryption key 

Finally, in step S22, the audio vault provides a message ID 
number and a crypto-key to the parties. The message ID and 
crypto-key can be used subsequently to retrieve the 
encrypted information recorded by the parties during the call 
While depicted as being provided at the end of the call, the 
message ID and crypto-key could also be provided at any 
time during the call. 

FIG. 3 depicts a message database 27a, which is used to 
describe each message stored by the audio vault. The fields 
of this database include a message ID number, the name and 
telephone number of the first calling party (who initiated the 
call), the telephone number of the second party, the date of 
the call, the message start time and end time, and a security 
level for the stored data. 

A unique message ID number is assigned to each stored 
message. The message ID number is used as an index into 
the message database 21a, as well as the message access 
database 21b and the message archive 27rf and it may be 
used to retrieve the message from the message archive. The 
message ID number also appears as an entry in the caller 
database 27c 

The remaining fields of the message database describe the 
various parameters associated with each stored message. 
The telephone number of all incoming calls may be 
extracted by the ACD using automatic number identification 
(ANI) . The telephone number of any call placed by the 
audio vault is, of course, known in advance. The name of the 
first party may be determined by using the first party's 
telephone number as an index into a look up table. 
Alternatively, it may be determined using caller ID or by 
asking the caller, via the I VRU, to input his name. The date, 
starting time, and ending time of the message are generated 
by clock 24 and stored in message database 21a. 

Different levels of security may be provided by the audio 
vault, and the security level for each recorded call is stored 
in the message database 27a. For example, one level of 
security could be digitally storing and encrypting the audio 
information. With this level of security, the caller must 
provide a crypto-key to retrieve and decrypt the stored 
message in addition to the message ID number. A higher 
level of security could be obtained if the key is only 
provided to a third party such as the court or an attorney. In 
this case, only the third party will be able to access the 
message once it has been stored by the user. Another level 
of security could allow access to the message only when one 
or more access codes are provided. Other levels of security 
can be readily envisioned. The security level field may also 
be used to describe the format of the encryption, if multiple 
encryption options arc provided. 

FIG. 4 shows a message access database 21b that includes 
a record for each recorded message, and is used to store the 
first and second party message access codes for each mes- 
sage. Every recorded message has an associated record in 
this database, indexed by the message ID number. The first 
party access code and the second party access code are 
stored. 

The access codes may be used to provide increased 
security for certain embodiments of the audio vault 12. In 
one embodiment, access to playback of the message will be 
granted if cither the first party access code or the second 
party access code is provided to the audio vault. Access to 
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modify the message, however, will only be granted if both 
the first party access code and the second party access code 
are provided. 

Other embodiments of the audio vault do not require a 
5 message access code to access the message, and will allow 
access using only the crypto-key and the message ID num- 
ber. Other message access arrangements can be readily 
envisioned. 

While the various message access arrangements are 

10 described above in terms of alternative embodiments, the 
audio vault can be implemented to allow a different access 
option for each stored message on an individual basis. The 
security level field in the message database may be used to 
store the message access option to be used for each message. 

15 Alternatively, an additional field may be added to the 
message access database 21b for this purpose. 

FIG. 5a depicts the fields of a caller database 27c which 
is used to index all messages by individual callers or 

2Q customers. This database includes fields for the name of the 
caller and a caller identification number that is uniquely 
assigned to each caller. The ANI field stores the telephone 
number of the caller, and it can be used to identify a caller 
from the ANI information received from incoming calls. 

25 Finally, a list of all the message ID numbers associated with 
each caller is also stored in the caller database 27c. This 
database can be accessed by caller name to provide a list of 
messages that belong to a given caller. It can also be 
accessed by message ID number to determine the name of 

3Q the caller for any given message. 

FIG. Sb depicts the message archive 27^ that is used to 
store the messages themselves. The message archive has a 
record for each message, which allows the message to be 
retrieved using the message ID number. Although the mes- 

35 sages depicted in the figure are short, longer messages may 
also be stored in the message archive, as the primary 
function of the message archive is to store the digitized and 
encrypted audio information (i.e., the contents of a 
conversation). Optionally, additional data may be stored in 

40 the message archive 21d together with the message, either 
embedded in the message, or in separate fields. This addi- 
tional data could include, for example, any of the data that 
is stored in the databases described above. 
Alternatively , the message may be stored directly in the 

45 message database, and the message archive can be omitted. 
FIGS. 7A and 7B depict the process of establishing the 
connections between the parties and the audio vault and the 
processing of caller information by the audio vault. Where 
a party places a call to the audio vault service, the call 

50 arrives, as an incoming call at call center PBX 26a associ- 
ated with the audio vault in step S30. The PBX extracts the 
telephone number of the incoming call using automatic 
number identification (ANI). In step S32 the PBX 26a stores 
the caller's number locally in caller database 27c and 

55 decides how to process the call based on rules programmed 
in the ACD. Next, in step S34, the audio vault obtains 
information about the incoming caller. This is accomplished 
by connecting the call to an IVRU 26c. The IVRU 26c 
prompts the caller with questions regarding the purpose of 

60 the call, and the responses to these questions determine the 
functions to be performed by the audio vault 12. For 
example, the IVRU 26c can request information about the 
" number of parties to be included in the call, and the level of 
security to be associated with the call. The IVRU 26c may 

65 also request personal information about the first party (the 
party who initiated the call), the telephone number of the 
other party to be included in the call, and optional additional 
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information about the other party. In step S36, the IVRU 26c 
captures the response of the first party to the questions asked 
in step S34. The responses to these questions are processed 
by the ACD, where it is determined what action must be 
taken in order to fulfill the terms of the caller. Steps S34 and 
S36 may be repeated as many times as necessary when 
multiple pieces of information must be obtained. The IVRU 
26 would thus prompt the caller for the first piece of 
information, then receive the first information. 
Subsequently, the IVRU 26c would prompt for and receive 
a second piece of information. This process continues until 
the IVRU has captured all of the required information. 

In step 38, the ACD 26b instructs PBX 26a to forward the 
incoming call to a holding queue. The personal information, 
the second party's phone number, and the purpose of the call 
are all stored in the appropriate record and remain logically 
linked with the call. ITie call data is then transferred to the 
central controller. In step S40, the ACD 26b instructs the 
PBX 26a to place a call to the second party, and the PBX 26a 
initiates the call in step S42. When the call is connected, it 
is routed to the IVRU 26c which will extract the appropriate 
information from the second party (similar to the informa- 
tion extracted from the first party). In step S44, the ACD 26b 
takes the information received from the second party and 
stores it locally in a caller database 27c. The ACD also 
transmits this information to the audio vault central control- 
ler. After processing the information from the second caller, 
in step S46 ACD 26b conferences the call from the first party 
and the second party together. Communication between the 
parties can then proceed as it would with an ordinary 
telephone conversation. In step S48, ACD 26b connects this 
two-party conference call to the audio vault central control- 
ler via the communication port 25. At this point, the audio 
vault 12 monitors the conversation between the two parties 
via the communication port 25, but does not record the 
conversation. 

As an alternative to having the audio vault initiate the 
second call (as depicted in step S40), the audio vault can 
wait for a second incoming call to arrive. With this 
arrangement, the parties must agree between themselves to 
call the audio vault at the same time, so that their calls can 
be connected and recorded by the audio vault. When this 
arrangement is used, the incoming calls may be matched 
with one another using a prearranged identifier that is 
extracted via the IVRU 26c. 

FIG. 8 depicts the initiation of the recording process, 
assuming an ongoing conversation has already been set up 
by the audio vault. In step S50, the communication between 
the second parties reaches a point where they wish to begin 
recording their conversation. Either party can initiate the 
recording by sending a signal to the audio vault 12, in step 
S52, requesting the audio vault to begin recording. This may 
be accomplished by pressing an appropriate key on a touch 
tone telephone keypad. The IVRU 26c will capture thC| 
keystroke and send it to the ACD 26b. In step S54, ACD 26b 
receives the digits captured by the IVRU 26c. Of course, 
instead of using a keystroke on a touch tone phone, other 
methods may be used to initiate the recording. For example, 
the audio vault may recognize for the phrase "begin record 
ing" using voice recognition, rjumero us other methods for 
acUvatin gth e record ing fii net ion ol tne audirrVa'Ult"wiirb*e 
apparenrtoThose skUlecHnTtie*"art. 

In step S56, the ACD 26b commands the IVRU 26c to 
play a pre-recorded notice indicating that recording will 
begin shortly, and asking the parties to consent to the 65 
recording. In step S58, the parties indicate that they agree to 
have their conversation recorded by either pressing an 



40 



45 



50 



55 



60 



appropriate button on the touch tone phone or by a voice 
response similar to the voice response described above. In 
step S60, the IVRU 26c captures the response of the parties 
and forwards it to ACD 266. If they have so agreed, the ACD 
26b then notifies the central controller that the parties have 
agreed to have their conversation recorded. Permission to 
record could take place earlier, be built into registration, or 
skipped altogether if laws permit. 

In step S62, the central controller CPU 21 checks the 
clock 24 and determines the exact starting time of the 
recording. The central controller then stores this starting 
time in the message database 27a. In step S64, the CPU 21 
assigns a message ID number to the call. It also creates a 
new record in the message database 276, which can be 
accessed using the message ID number. Then, in step S66, 
recording of the call begins. 

FIG. 9 depicts the process of recording information in the 
audio vault. In step S70, the audio content of the conversa- 
tion is received via a caller interface such as IVRU 26c 
coupled to ACD 26b y and transmitted to the CPU 21 by the 
ACD 26b. In step S72, the audio message signal, which until 
now has been maintained in an analog format, is converted 
to digital data by CPU 21. In step S74, the digitized audio 
message data is encrypted by cryptographic processor 28. 
The date and the exact time of the call is embedded in the 
digitized audio data. In step S76, the encrypted digital audio 
content of the conversation is stored in the message archive 
21 d. This step continues as long as additional audio infor- 
mation continues to arrive from the parties. Additional time 
stamps, similar to the one embedded in step S74, may be 
added to the audio data at various intervals. Further, caller 
information such as name, telephone number, caller ID and 
the like, can be incorporated into the encrypted message to 
provide an additional level of security. 

Referring now to FIG. 10, when a party wishes to stop the 
recording, the party presses a key on a touch tone telephone, 
as depicted in step S80. In step S82, IVRU 26c will capture 
this touch tone command and ACD 26b will notify the 
central controller that the recording should be terminated. 
Likewise, if the line or connection is dropped, the recording 
is terminated. In step S84, the central controller receives the 
final portion of the audio transmission, and then completes 
the encryption of the digitized message. Optionally, an 
additional time stamp may be embedded in the audio infor- 
mation at this point. In step S86, the CPU 21 instructs the 
cryptographic processor 28 (shown in FIG. 2) to generate a 
symmetric key. In step S88, the cryptographic processor 28 
generates a symmetric key that will ultimately be used to 
decrypt the encrypted information. In step S90, the CPU 21 
transmits the symmetric cryptographic key to the IVRU 26c. 
The CPU 21 also transmits a message ID number to the 
IVRU 26c. Finally, in step S92, the symmetric key and the 
message ID number are distributed by having the IVRU 26c 
provide them in audio format to the parties participating in 
the call. 

In the embodiment described above, one key and message 
ID number is provided to each of the parties to the conver- 
sation. Any party to the conversation can subsequently use 
the key and the message ID number to retrieve the conver- 
sation. In one embodiment, in order to maintain the integrity 
and security of the message, none of the parties may 
authorize the deletion or modification of the recorded mes- 
sage. 

In an alternative embodiment modifications of the mes- 
sage can be requested by the participants in the conversation. 
This embodiment uses a set of access codes in addition to the 
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message ID and decryption key. Each party to the conver- Selective access can be implemented by providing a 

sation receives a unique access code, and any given party unique crypto-key and access code to each participant, 

does not know the access codes of the other parties. Only which will enable him to access only his customized copy of 

when all of the access codes have been collected by the the conversation. 

audio vault will the audio vault permit the modification of a 5 while the embodimenls desc ribed above involve two or 

recording. It should be noted that in this embodiment, the t , „ „ , t . u t t . 

& t , , Tr * ■ i i , c 4 . three callers connected to the audio vault on separate tele- 
same crypto-key and message ID are provided to each of the . . ... 4 , m j j 
parties, and any party can obtain playback of the conversa- P hone hnes ' th u ese embodiments can be eastly extended to 
tion by providing the message ID, the crypto-key, and their ^ number of calle[S connected on separate telephone 
unique access code. The system may also be configured to lines. 

allow playback using only the crypto-key and the message An alternative embodiment using only one telephone line 

ID, without providing an access code. This may be useful, may also be implemented, as depicted in FIG. ID. This 

for example, when an employer wants his employees to have embodiment may be used by individuals to unilaterally 

access to play a recording only, without authorizing the record information in a secure manner. The operation is as 

employees to modify the recorded information. described above, the message received from the simple 

Another embodiment of this invention uses two crypto- 15 connection and only one message ID and crypto-key being 

keys and a single message ID for each recorded conversa- generated. The system could be used, for example, to record 

tion. The message is encrypted so that either of the two keys a last w iU and testament. Optionally, the crypto-key could be 

will enable the audio vault to decrypt the recording. The provided to a trusted third party such as an attorney. The 

message ID plus a one of the two crypto keys arc distributed system could also be used to record an invention disclosure 

to each party. When the system is implemented in this lo P rove that a Person conceived of an invention before a 

manner, any party can play back a message by providing his S iven date - 0tner applications can be readily envisioned, 

decryption key and the message ID to the audio vault. When The single telephone line embodiment can also be used to 

both crypto-key s are received, the system will also authorize record two-party conversations if the parties to an ordinary 

modification or deletion of the recorded message. No access 25 telephone conversation place a conference call to the audio 

codes are needed in this embodiment. vault. The audio vault would then receive this conference 

Yet another embodiment of this invention is provided for call on a single telephone line. In this embodiment, however, 
use where multiple parties participate in different sections of the audio vault will not be able to automatically determine 
a single conversation. In this embodiment, the audio vault the identity of all of the parties. Accordingly, the audio vault 
provides each party with access to only those portions of the 30 can query the parties and ask them to provide their telephone 
conversation in which he participated. For example, a con- numbers or other identifying information using the IVRU 
versation may occur in which party A speaks with party B capabilities. The audio vault could identify the parties using 
for one minute. Then, party C is conferenced into the call, a voice recognition system, or only provide one crypto-key 
and the conversation continues for an additional minute. a nd message ID for use by all of the callers. 
Next, party B hangs up, and parties A and C continue to 35 As yet another alternative embodiment, also depicted in 
speak for an additional minute. In this example, party A was FIG. ID, the audio vault may be implemented as a stand- 
present during the whole conversation, party B was present alone device that is not connected to a telephone line. In this 
for only the first two minutes, and party C was present for embodiment, the audio vault will resemble a traditional 
only the last two minutes. This is represented schematically digital audio recorder but will further include a crypto- 
in FIG. 11, where the time axis represents time in minutes. 40 graphic processor, input means (such as a keyboard or a 

By storing a digitized encrypted copy of the conversation voice recognition unit) for receiving identification informa- 

customized for each participant, the audio vault can selec- tion from the users, and output means for providing cryp- 

tively provide playback access to only those portions of the tographic keys to the user. The output means could include, 

conversation in which a given party participated. In this for example, a speech synthesis circuit or an alphanumeric 

example, because party A participated during the whole 45 display. This stand-alone audio vault can also embed time 

conversation, A's copy will contain all three minutes of the stamps into the digitally stored and encrypted information, 

conversation. Because party B participated in only the first The time stamping information may be derived locally from 

two minutes of the conversation, B 5 s copy will contain only a n internal, tamper proof clock. Alternatively, the time stamp 

those two minutes. Similarly, because party C participated in information may be received via radio transmissions from a 

only the last two minutes of the conversation, C's copy will 50 remote location. The system could further obtain location 

contain only those two minutes. information from a global positioning satellite (GPS) system 

In a further extension of this embodiment, the audio vault or DV triangulating cellular phone signals, and this location 

12 stores the input received from each individual line as a information can be recorded along with the audio inform a- 

separatc message with its own message ID. Thus, the words ^ on - ^ 

produced by each participant arc stored separately. In a 55 Numerous modifications to the embodiments described 

conversation between four people, it would thus be possible above can be readily envisioned. For example, audio vault 

for one party to hear the conversation of three participants, may be implemented on an internal corporate telephone 

but be precluded from listening to the fourth participant. system. Or instead of using telephone lines to connect the 

Using the example depicted in FIG. 11, there are three users to the server, a computer network connection may be 

separate lines coming into the audio vault 12. Each line has 60 used to link parties that own computers. In a further 

its own unique stream of digital data being stored in separate embodiment, the present invention may also be imple- 

records in the message archive. If party B wants access to his mented by connecting to the parties over the Internet, where 

segment of the conversation, the Audio vault 12 retrieves the communications are transmitted in packets, 

appropriate segment and provides the data to him. In another embodiment, the timestamp associated with 

If party B wants to hear the input from party A and party 65 each stored message also includes representations of one or 

C as well, they would need to provide the necessary access more previous message timcstamps, lo provide an additional 

means for him to listen to their segments of the conversation. degree of message timestamp assurance. For example, a 
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hash value of the last three timestamps can be stored in 
memory for incorporation into the current timestamp. The 
hash values are calculated by applying a hash algorithm to 
the cleartext timestamps. The following example illustrates 
this technique. Four messages are received and stored by the 5 
audio vault with the first message stored at nine hours, 
thirty-one minutes and twenty seconds ("09:31:20"). The 
second, third, and fourth messages are stored at 09:31:50, 
09:32:10, and 09:32:30, respectively. The timestamp hash 
value associated with the fourth message received is com- 10 
puted as follows: 

Fourth Message Timestamp Hash Value-Hash (09:31 :2Q)+Hash 
(09:31 :50)+Hash (09:32 :10)+Hash (09:32:30) 

Thus, the hash values for each message relate to their 
respective previous three messages. Such hash chaining 
discourages fraudulent modification of timestamps. 

Suppose a forger discovers the private key used to encrypt 
the timestamp of the message stored at 09:31:50 and uses it 
to change both the cleartext and hashed parts of the times- 
tamp. A suspicious party could then challenge the integrity 
of the 09:31:50 timestamp by recomputing the appropriate 
timestamp hash values of the subsequent three stored mes- 
sages. If the recomputed hash values do not match the 
expected hash values, the 09:31:50 timestamp is demon- 
strated to have been altered. When tampering is generally 25 
suspected but no specific timestamp is in question, an altered 
timestamp can be determined by recomputing the most 
recent stored timestamp and continuing backwards until 
three successive incorrect timestamps are found. Of course, 
the forger could theoretically change all the timestamps in 30 
the chained hash, but this would require more effort than 
changing just the desired one, and would increase the 
chances of detection. 

In addition to hashing the timestamp associated with each 
message, the audio vault may compute the hash value of the 35 
stored audio signals, incorporating the hash values into the 
timestamp of subsequent messages, or even into the stored 
audio of subsequent messages. Attempts to alter a message 
would therefore be evident from the recalculation of subse- 
quent hash values. The uses and advantages of hash func- 
tions are discussed generally in Schneier, "Applied Cryp- 40 
tography" (2d ed. 1996), chapter 18. Suitable conventional 
hash algorithms include the Secure Hash Algorithm (SHA) 
developed by the National Institute of Standards and Tech- 
nology. 

Certain well-known enhancements to public key cryp tog- 45 
raphy can also be used to provide greater security. For 
example, the message could include a digital certificate for 
public key distribution to a parties that do not know the 
message public key needed to verify a timestamp encrypted 
with the message private key. In such a digital certificate, the 50 
message public key is encrypted (and vouched for) by the 
private key of a trusted whose public key is known to the 
recipient. The recipient uses the certifier's public key to 
decrypt the message public key, then uses the message 
public key to verify the timestamp. Alternatively, the recipi- 55 
ent could simply obtain the message public key from a 
publicly accessible database, eliminating the need for digital 
certification. 

To provide an additional measure of security, digital 
v signatures may be generated and stored in the message 
database, or alternatively embedded in the audio information 60 
stored in the message archive. 

To this point, asymmetric (public key) encryption has 
been discussed in the context of the various cryptographic 
operations. However, symmetric key (e.g., DES) key 
encryption is also possible, either as a replacement for, or 65 
adjunct to (e.g., a symmetric session key transmitted using 
public cryptography) public key cryptography. 
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The uses and advantages of digital signatures are dis- 
cussed generally in Schneier, "Applied Cryptography" (2d 
ed. 1996), chapter 2. 

By providing a system as described above, the audio vault 
can maintain audio records that are authenticatable as to 
both time and content of a conversation. 

While the invention has been described above in terms of 
specific embodiments, it is to be understood that the inven- 
tion is not limited to the disclosed embodiments. On the 
contrary, this invention is intended to cover various modi- 
fications and equivalent structures included within the spirit 
and scope of the appended claims. 

We claim: 

1. A process for recording audio communications, com- 
prising: 

establishing an audio connection with a calling party; 

receiving an audio communication over the audio con- 
nection from the calling party; 

encrypting the audio communication to provide an 
encrypted audio communication; 

storing the encrypted audio communication; and 

providing to the calling party a code for decrypting the 
encrypted audio communication. 

2. A process for recording audio communications, com- 
prising: 

establishing an audio connection with at least two parties; 
accessing an audio communication between the at least 
two parties; 

encrypting the audio communication to provide an 

encrypted audio recording; 
generating a key which can be used to decrypt the 

encrypted audio recording; 
embedding a time stamp in the encrypted audio recording; 
generating at least two access codes, any of which can be 

used to obtain access to the encrypted audio recording; 
transmitting the key to the at least two parties; and 
transmitting the at least two access codes respectively to 

the at least two parties such that each party receives a 

unique access code. 

3. A process for recording audio communications, com- 
prising: 

establishing an audio connection with at least two parties; 
accessing an audio communication between the at least 
two parties; 

encrypting the audio communication to provide an 

encrypted audio recording; 
generating a key which can be used to decrypt the 

encrypted audio recording; 
generating at least two access codes, any of which can be 

used to obtain access to the encrypted audio recording; 
transmitting the key to the at least two parties; 
transmitting the at least two access codes respectively to 

the at least two parties such that each party receives a 

unique access code; 
receiving a telephone call from one of the at least two 

parties; 

receiving one of the at least two access codes from the one 

of the at least two parlies; 
receiving the key from the one of the at least two parties; 
using the one of the at least two access codes to gain 

access to the encrypted audio recording; and 
using the key to decrypt the encrypted audio recording. 
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